Past Event: Babuška Forum

Error measures for black box models that fit to the ground truth on real data

Adam Sobey, Programme Director for Data-Centric Engineering at The Alan Turing Institute and Chair of Data-Centric Engineering in the Maritime Engineering Group at The University of Southampton

Abstract

Machine Learning is becoming increasingly prevalent across engineering sectors. These approaches are allowing us to build data models in areas where traditional modelling and simulation is currently impossible, with benefits such as increased safety or reduced emissions. However, as we increase the range of applications for these tools, we are applying them on datasets that don’t fit to the assumptions required to generate accurate input-output relationships, the ground truth. This limits the potential for these methods and means that optimisation is impossible. A key problem is the error measures that we use, all of which are derived from the same Minkowski family. This talk will outline some of these problems and the selection of the Fit to Median Error measure for machine learning regression automation. The networks optimised for their Fit to Median Error are shown to approximate the ground truth more consistently, without sacrificing conventional Minkowski error values. A ship power prediction model is considered, which is highly stochastic in nature, and where more consistent input-output relationships allow a potential 18% reduction in fuel usage.

Biography

Adam Sobey is Programme Director for Data-Centric Engineering at The Alan Turing Institute and Chair of Data-Centric Engineering in the Maritime Engineering Group at The University of Southampton. He completed his MEng in Aerospace Engineering with Astronautics in 2006 before moving to the Maritime Engineering group to do a PhD focusing on implementing AI approaches into Leisure Boatbuilding in 2010. He joined the group as a Research Fellow in 2009 to lead the LR/MoD Centre of Excellence in Marine Structures before moving to LRET funding to explore the use of Multi-Level Selection mechanisms within Evolutionary Computation. Between 2014-2017 he was a visiting Scientist at the Institute of High Performance Computing before returning to take up a lectureship at the University of Southampton 2018, Associate Professor 2020 and Professorship 2022. In 2019 he was asked to start the Marine and Maritime group within the Data-Centric Engineering programme at The Alan Turing Institute, before becoming director in 2023. He received the Royal Institute of Naval Architects Jeom Paik award for best Paper on Structures under 30 in 2015 for work related to improving Structural Safety of Composite Ships. He is currently an Editor of Data-Centric Engineering and Associate Editor of Ship and Offshore Structures. He is also the Director of the Centre of Excellence of Data-centric Engineering for Clean Oceans at the University of Southampton, bringing together experts from across the University to provide a real world difference in decarbonisation.